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Abstract 



Given two bounded convex sets X C R m and F C R™, specified by membership ora- 
cles, and a continuous convex-concave function F : X x Y — > R, we consider the problem 
of computing an e-approximate saddle point, that is, a pair (x*,y*) <E X x Y such that 
sup y£Y F(x* , y) < inf Te A" F(x, y*) + e. Grigoriadis and Khachiyan (1995), based on a ran- 
| domized variant of fictitious play, gave a simple algorithm for computing an e-approximatc 

C/5 ■ saddle point for matrix games, that is, when F is bilinear and the sets X and Y are sim- 

i plices. In this paper, we extend their method to the general case. In particular, we show 

that, for functions of constant "width", an e-approximate saddle point can be computed 
£NJ | using O* (n + m) random samples from log-concave distributions over the convex sets X 

and Y. As a consequence, we obtain a simple randomized polynomial-time algorithm that 
| computes such an approximation faster than known methods for problems with bounded 

■ width and when e <G (0, 1) is a fixed, but arbitrarily small constant. Our main tool for 

| achieving this result is the combination of the randomized fictitious play with the recently 

developed results on sampling from convex sets. 

o ' 

1 Introduction 

Let X C R m and Y C R n be two bounded convex sets. We assume that each set is given by a 
membership oracle, that is an algorithm which given x E W 71 (respectively, y G W 1 ) determines, 
in polynomial time in m (respectively, n), whether or not x € X (respectively, y G Y). Let 
FilxF-^Rbea continuous convex-concave function, that is, F(-, y) : X — > R is convex for 
all y £ Y and F(x,-) : y — )• R is concave for all x G X. We assume that we can evaluate F at 
rational arguments in constant time. The well-known saddle-point theorem (see e.g. |Roc70j ) 
states that 

v* = inf sup F(x,y) = sup inf F(x,y). (1) 

This can be interpreted as a 2-player zero-sum game, with one player, the minimizer, choosing 
her/his strategy from a convex domain X, while the other player, the maximizer, choosing 
her/his strategy from a convex domain Y. For a pair of strategies x G X and y EY, F(x,y) 
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denotes the corresponding payoff, which is the amount that the minimizer pays to the maxi- 
mizes An equilibrium, when both X and Y are closed, corresponds to a saddle point, which 
is guaranteed to exist by JT]), and the value of the game is the common value v*. When an 
approximate solution suffices or at least one the sets X or Y is open, the appropriate notion 
is that of e-optimal strategies, that a pair of strategies (x*,y*) £ X xY such that for a given 
desired absolute accuracy e > 0, 

sup F(x*,y) < inf F(x,y*) + e. (2) 

y € Y X ^ X 

There is an extensive literature on the existence of saddle points in this class of games and 
generalization (see e.g. |Dan631 IGro671 ltKP901 IMcL841 IVor84l IRoc701 ISha581 ITer721 IWal451 
ISeb90| ) and their applications (see e.g. |Bel97l IDKR911 IWas03| ). A particularly important 
case is when the sets X and Y are polytopes with an exponential number of facets arising as 
the convex hulls of combinatorial objects (see Appendix El for some applications). 

One can easily see that ([T|) can be reformulated as a convex minimization problem over 
a convex set given by a membership oracl^E and hence any algorithm for solving this class 
of problems, e.g., the Ellipsoid method, can be used to compute a solution to ([2]), in time 
polynomial in the input size and polylog(^). However, there has recently been an increasing 
interest in finding simpler and faster approximation algorithms for convex optimization prob- 
lems, sacrificing the dependence on e from polylog(-) to poly(-), in exchange of efficiency in 
terms of other input parameters; see e.g. [AHK051 IAK071 IBBR041 IGK921 IGK951 IGK981 IGK041 
IKha04l lKa!07l ILTOI IKY071 IYouOH IPST9T] and [GK941 IGK961 IGKPV011 iJaTIM IDJ07] . 

In this paper, we show that it is possible to get such an algorithm for computing an e- 
saddle point (J2j). Our algorithm is based on combining a technique developed by Grigoriadis 
and Khachiyan }GK95| . based on a randomized variant of Brown's fictitious play [Bro51] . with 
the recent results on random sampling from convex sets [LV061 IVem05] . Our algorithm is 
superior to known methods when the width parameter p (to be defined later) is small and 
s G (0, 1) is a fixed but arbitrarily small constant. 

2 Our Result 

We need to make the following technical assumptions: 

(Al) We know £° G X, and 77° G Y, and strictly positive numbers rx, Rx, fy, and Ry 
such that B m (£°,r x ) CXC B m (0,R x ) and B n {rf ) ,r Y ) C Y C B n (0,Ry), where B k (x°,r) = 
{x G M. k : \\x — x°\\2 < r} is the A:-dimensional ball for radius r centered at x° G M k . In 
particular, both X and Y are full-dimensional in their respective spaces (but maybe open). In 
what follows we will denote by R the maximum of {Rx,Ry, 7^7 }• 

(A2) \F(x, y)\<l for all x G X and y G Y. 

Assumption (Al) is standard for algorithms that deal with convex sets defined by member- 
ship oracles (see, e.g., |GLS93j ). and will be required by the sampling algorithms. Assumption 
(A2) can be made without loss of generality, since the original game can be converted to 
an equivalent one satisfying (A2) by scaling the function F by ^, where the "width" pa- 
rameter is defined as p = max^gx^eY \F( x i u) I- (For instance, in case of bilinear function, 
i.e, F(x,y) = xAy, where A is given m x n matrix, we have p = max x& x,yeY \ x Ay\ < 
yJmnRxRy max{|ajj| : i G [m], j G [n]}.) Replacing e by |, we get an algorithm that 
works without assumption (A2) but whose running time is proportional to p 2 . We note that 

1 Minimize F(x), where F(x) = maxj, F(x,y). 



2 



such dependence on the width is unavoidable in algorithms that obtain e-approximate so- 
lutions for negative functions and whose running time is proportional to poZy(-) (see e.g. 
[AHK 06, PS T9I] ); otherwise, scaling would yield an exact algorithm for rational inputs. For 
nonnegative functions F, the dependence on the width can be avoided, if we consider relative 
approximation errors (see [GK981 iKhaM IKY071 lYouOlj ^. 

We assume throughout that e is a positive constant less than 1. 

The main contribution of this paper is to extend the randomized fictitious play result in 
[GK95j to the more general setting given by ([2]). 

Theorem 1 Assume X and Y satisfy assumption (Al). Then there is a randomized algorithm 
that finds a pair of e- optimal strategies in an expected number of 0{ p ("+ m ) l n i^) iterations, 
each computing two approximate samples from log-concave distributions. The alaorithrfi^ 
requires 0*( p ( n + m ) l n -^) calls to the membership oracles for X and Y. 

When the width is bounded and e is a fixed constant, our algorithm needs 0*((n + m) e lnR) 
oracle calls. This is superior to known methods, e.g., the Ellipsoid method, that compute the 
e-saddle point in time polynomial in log -. In Appendix[Al we give examples of problems with 
bounded width arising in combinatorial optimization. 



3 Relation to Previous Work 



Matrix and polyhedral games. The special case when each of the sets X and Y is a 
polytope (or more generally, a polyhedron) and payoff is a bilinear function, is known as 
polyhedral games (see e.g. |Was03j ) . When each of these polytopes is just a simplex we obtain 
the well-known class of matrix games. Even though each polyhedral game can be reduced 
to a matrix game by using the vertex representation of each polytope (see e.g. |Sch86| ). this 
transformation may be (and is typically) not algorithmically efficient since the number of 
vertices may be exponential in the number of facets by which each polytope is given. 

Fictitious play. We assume for the purposes of this subsection that both sets X and Y 
are closed, and hence the infimum and supremum in (pQ) are replaced by the minimum and 
maximum, respectively. 

In fictitious play which is a deterministic procedure and originally proposed by Brown 
|Bro51j for matrix games, each player updates his/her strategy by applying the best response, 
given the current opponent's strategy. More precisely, the minimizer and the maximizer ini- 
tialize, respectively, x(0) = and y(0) = 0, and for t = 1, 2, . . . , update x(t) and y(t) by 

x(t + 1) = j^x(t) + wh ere £(t) = argmin^ F(£, y(t)), (3) 

y(t + 1) = j-p^yif) + J^jVit), where r)(t) = argmax^y F(x(t),n). (4) 

For matrix games, the convergence to optimal strategies was established by Robinson [Rob51] . 
In this case, the best response of each player at each step can be chosen from the vertices of 

the corresponding simplex. A bound of I — - — I on the time needed for convergence to 

2 See Section [5731 for details. 

3 Here, we apply random sampling as a black box for each iteration independently; it might be possible to 
improve the running time if we utilize the fact that the distributions are only slightly modified from an iteration 
to the next. 

4 0*( ) suppresses polylogarithmic factors that depend on n, m and e. 
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an e-saddle point was obtained by Shapiro [Sha58j . More recently, Hofbauer and Sorin |HS06] 
showed the convergence of fictitious play for continuous convex-concave functions over compact 
convex sets; they also gave a dynamic proof for the minmax theorem. 

Randomized fictitious play. In [GK95] . Grigoriadis and Khachiyan introduced a random- 
ized variant of fictitious play for matrix games. Their algorithm replaces the minimum and 
maximum selections Q-dH by a smoothed version, in which, at each time step t, the min- 
imizing player selects a strategy i G [m] with probability proportional to exp { — |ejAy(t)}, 
where denotes the ith unit vector of dimension m. Similarly, the maximizing player chooses 
strategy j G [n] with probability proportional to exp {^x(i)Aej}. Grigoriadis and Khachiyan 
proved that, if A G [— l,l] mxn , then this algorithm converges, with high probability, to an 
e-saddle point in 0( log (™ +n ) ) iterations. Our result builds on |GK95j . 

The multiplicative weights update method. Freund and Schapire JFS99J showed how to 
use the weighted majority algorithm, originally developed by Littlestone and Warmuth |LW94] . 
for computing e-saddle points for matrix games. Their procedure can be thought of as a 
derandomization of the randomized fictitious play described above. Similar algorithms have 
also been developed for approximately solving special optimization problems, such as general 
linear programs [PST91J, multicommodity flow problems |GK98j . packing and covering linear 
programs [PST914 IGK98} IGK041 IKY071 lYouOl] . a class of convex programs [Kha04| . and 
semidefinite programs jAHK05( IAK07| . Arora, Hazan and Kale [AHK06| consider the following 
scenario: given a finite set X of decisions and a finite set Y of outputs, and a payoff matrix 
M G R^**' such that M(x, y) is the penalty that would be paid if decision x G X was made and 
output y G Y was the result, the objective is to develop a decision making strategy that tends 
to minimize the total payoff over many rounds of such decision making. Arora et al. |AHK06[ 
IKal07j show how to apply this framework to approximately computing max yg y minj g [ m ] fi(y), 
given an oracle for finding max^gy X)ie[m] f° r an y non-negative A G R m such that 

Y^iLx = 1, where Y C R n is a given convex set and /i, . . . , f m : Y — > R are given concave 
functions (see also |Kha04] for similar results). 

There are two reasons why this method cannot be (directly) used to solve our problem 
([2]). First, the number of decisions m is infinite in our case, and second, we do not assume 
to have access to an oracle of the type described above; we assume only a (weakest possible) 
membership oracle on Y. Our algorithm extends the multiplicative update method to the 
computation of approximate saddle points. 

Hazan's Work. In his Ph.D. Thesis |Haz06| . Hazan gave an algorithm, based on the multi- 
plicative weights updates method, for approximating the minimum of a convex function within 
an absolute error of e. Theorem 4.14 in |Haz06j suggests that a similar procedural can be used 
to approximate a saddle point for convex-concave functions, however, without a running time 
analysis. 

Sampling algorithms. Our algorithm makes use of known algorithms for sampling from 
a given log-concave distribution^ /(•) over a convex set X C R m . The currently best known 
result achieving this is due to Lovasz and Vempala (see, e.g., |LV07l Theorem 2.1]): a random 
walk on X converges in 0*(m 5 ) steps to a distribution within a total variation distance of e 
from the desired exponential distribution with high probability. 

5 This procedure can be written in the same form as our Algorithm [1] below, except that it chooses the points 
£ X and rj(t) £ Y, at each time step t — 1, . .. ,T, as the (approximate) centroids of the corresponding 
sets with respect to densities p s (t) = e^iMe-fK.iW)) and q ^ = e £t= 1 iM<=+*'(fM,i7)) ( both f which are 
log-concave distributions), and outputs 2t=i x (t)< T Stli 3/W) a ^ the end. 

6 that is, log/(-) is concave 



4 



Algorithm 1 Randomized fictitious play 



Input: Two convex bounded sets X,Y and a function F(x,y) such that F(-,y) : X — > R is 
convex for all y G V and -F(cc, •) : Y — > M is concave for all a; G X, satisfying assumptions 
(Al) and (A2) 
Output: A pair of e-optimal strategies 
1: t := 0; choose x(0) G X; y(0) G 1", arbitrarily 
2: while t < T do 

3: Pick (el and 77 G Y, independently, from X and Y with densities n|f|nH an d flgMjR ' 
respectively 

4: x(i + 1) := £-x{t) + y(t + 1) := ^y{t) + ^77; t := t + 1; 

5: end while 

6: return (x(t),y(i)) 



Several algorithms for convex optimization based on sampling have been recently proposed. 
Bertsimas and Vempala |BV04j showed how to minimize a convex function over a convex set 
X C R m , given by a membership oracle, in time 0*((m 5 C + m 7 ) log R), where C is the time 
required by a single oracle call. When the function is linear time 0*(m 4 ' 5 C) suffices (Kalai 
and Vempala |KV06j ). 

Note that we can write (pQ) as the convex minimization problem ini x ^x F(x), where F(x) = 
su Py£Y F{x, y) is a convex function. Thus, it is worth comparing the bounds we obtain in The- 
orem [1] with the bounds that one could obtain by applying the random sampling techniques 
of [BV041 IKV06| (see Table 1 in |BV 04] for a comparison between these techniques and the 
Ellipsoid method). Since the above program is equivalent to inf{f : x G X, and F(x,y) < 
v for all y G Y}, the solution can be obtained by applying the technique of |BV041 IKV06] . 
where each membership call involves another application of these techniques (to check if 
su PyeY F{ x i v) — v )- The total time required by this procedure is 0*(n 4,5 (m 5 C + m 7 ) log R), 
which is significantly greater than the bound stated in Theorem [TJ 

4 The Algorithm 

The algorithm proceeds in steps t = 0, 1, . . ., updating the pair of accumulative strategies x(t) 
and y(t). Given the current pair (x(t),y(t)), define 

etFle.ylt)) 

Pt(t) = e * for£GA, (5) 

etF(x(t),r]) 

q v (t) = e 2 for r] G Y, (6) 

and let 

IIpWIIi = / P&H and || g (t)||i = / q v (t)d7]. 

Our algorithm Q] is a direct generalization of the algorithm in |GK95j . The parameter T will 
be specified later (see Lemma H]). 

5 Analysis 

Following [GK95| . we use a potential function 3>(i) = ||p(t)||i||<z(£)||i to bound the number of 
iterations required by the algorithm to reach an e-saddle point. The analysis is composed of 
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three parts. The first part of the analysis (Section 5.1), is a generalization of the arguments in 
[GK95j (and |KY07| ) : we show that the potential function increases, on the average, only by a 
factor of e°( £ \ implying that after t iterations the potential is at most a factor of e°( e > t of the 
initial potential. While this was enough to bound the number of iterations by e -2 log(re + m) 
when both X and Y are simplices and the potential is a sum over all vertices of the simplices 
[GK95j . this cannot be directly applied in our case. This is because the fact that a definite 
integral of a non-negative function over a given region Q is bounded by some r does not imply 
that the function at any point in Q is also bounded by r. In the second part of the analysis 
(Section 5.2), we overcome this difficulty by showing that, due to concavity of the exponents 
in ([5]) and Q, the change in the function around a given point cannot be too large, and hence, 
the value at a given point cannot be large unless there is a sufficiently large fraction of the 
volume of the sets X and Y over which the integral is also too large. In the last part of the 
analysis (Section 5.3) , we show that the same bound on the running time holds when the 
sampling distributions in line [3] of the algorithm are replaced by sufficiently close approximate 
distributions. 



5.1 Bounding the potential increase 
Lemma 1 For t = 0, 1, 2, . . . , 



E[$(t + 1)] <E[*(t)](l + | 2 f. 

O 

Proof Conditional on the values of x(t) and y(t), we have 



l|p(* + i)lh 



e 2 <j£ 



Hex 



< 



etF(j,y(t)) e£(^v) 

e 2 e 2 d£ 



£6X 



p € (t)e 2 di 



< 



[ Pdt) 



e 2 e 



1 + J~2 F ^ 



using assumption (A2), concavity of F(£, ■) : Y — > R and the inequality e 5 < 1 + 5 + |5 2 , valid 
for all 5 E [— ojh]- Taking the expectation with respect to rj (with density proportional to 
q v (t)), we get 

E g [||p(t + l)||i]< 



1 + 

6 2 



l \PV l 



(7) 



Similarly, by taking the expectation with respect to ^ (with density proportional to p^(t)), we 
can derive 



Ep[lk(t + i)lli]<lk(t)lli 



(8) 
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Now, using independence of £ and 77, we have 

E[$(t + l)\x(t),y(t)} < *(*) 

ftexPd-J JneYWJ- 1 ysi'V""i"s 



2\ 2 
6 



6 



b(*)llilk(*)lli IkWIklbWIli ; 



e 2 fsexPttt) J V €Y Qr,(t)F(£,ri)d,rid£ J veY q v (t) f^ x Pt(t)F(£,ri)d£dri 



.1 

By interchanging the order of integration, we get that the second part of the sum on the 
right-hand side is zero, and third part is non-positive. Hence, 

E[*(t + l)|s(t),y(t)]<*(t)(l + ^ ■ (9) 

The lemma follows by taking the expectation of Q with respect to x(t) and y(t). □ 

By Markov's inequality we have the following statement. 
Corollary 1 With probability at least i , after t iterations, 

< 2e^*$(0). (10) 

At this point one might tend to conclude the proof, as in |GK95t IKY07] , by implying from 
Corollary [T] and the non- negativity of the function under the integral 

$(t) = f e it(F(x(t), V )-F(t, ymd {: dr)j (H) 

j£eX,rieY 

that this function is bounded at every point also by 2e~ t Q(0) (with high probability). This 
would then imply that the current strategies are e-optimal. However, this is not necessarily 
true in general and we have to modify the argument to show that, even though the value of 

e 2 

the function at some points can be larger than the bound 2e~3~*<3?(0), the increase in this value 
cannot be more than an exponential (in the input description), which would be still enough 
for the bound on the number of iterations to go through. 



5.2 Bounding the number of iterations 

For convenience, define Z = X x Y, and concave function gt : Z — > R given at any point 
z = (l;,r]) £ Z by </t(£, 77) := %t (F(x(t),r]) — F(£, y(t))). Note that, by our assumptions, Z is a 
full-dimensional bounded convex set in M. N of volume <£(()) = vol(X)-vol(Y), where N = n+m. 
Furthermore, assumption (A2) implies that |<7t(;z)| = \^t(F(x(t),rj) — F(£,y(t))) \ < et for all 
z€ Z. 

A sufficient condition for the convergence of the algorithm to an e-approximate equilibrium 
is provided by the following lemma. 

Lemma 2 Suppose that \10}) holds and there exists an a such that 

< a < 4et, (12) 
e* a (^) N vol(Z) > 1. (13) 
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Then 

e gt(z) < 2e 4 i+a $(0) for all z€ Z. (14) 

2 

Proof Assume otherwise, i.e., there is z* G Z with <7t(z*) > ^-t + a + ln(2<3?(0)). Let 

A* = a/(4et), 

Z + = {z£ Z| 5 t(z) > - a/2}, and Z++ = {z* + ±(z - z*)\z G Z+}. 

A* 

Concavity of gt implies convexity of Z + . This implies in particular that z* + X'(z — z*) G Z ++ 
of all < A' < -±- and z G Z + , since z* + A*A'(z — z*) G Z + . We claim that Z C Z ++ . Assume 
otherwise, and let x G Z\ Z ++ (and hence x £ Z \ Z + ). Let 

A+ = sup{A | z* + X(x - z*) G Z + } and z + = z* + X + (x - z*). 

By continuity of g t , z + G Z + and gt(z*) — a/2 = g t (z + ). We have x — z* = ;xr( z+ — z*) and 
hence pp- > j*. But z + = A + x + (1 — A + )z* and hence 

- a/2 = <fc(z+) = g t (X + x + (1 - A+)z*) > A+<? t (x) + (1 - X + )g t (z*). 

Thus 

|<A + ( 5t (z*)- 5t (x))<2ea + , 
a contradiction. We have now established Z C The containment implies 

N 



vol(Z) < vol{Z ++ ) = (^\ vol{Z + ) 



and further 



$(t) = / e^cZz > / e 9tW dz 

> 2$(0)eV i+ 2« TO /(Z + ) > 2$(0)eV'+2 Q uo i(z) > 2$(0)eV 



a contradiction to (jlOj) . □ 

We can now derive an upper-bound on the number of iterations needed to converge to 
e-optimal strategies. 

Lemma 3 If (T^P holds and 

t > ^(a + max{0,ln(2uoZ(Z))}), (15) 



then (x(t),y(t)) is an e-optimal pair. 

3 



Proof By dHJ) we have g t (z) < ^-t + a + ln(2$(0)) = ^t + a + ln(2t>o/(Z)) for all z G Z, or 



equivalently, 



e e 2 

-t(F(x(t), V ) -F(£,y(t))) < —t + a + \xi(2vol(Z)) for all £ G X and rj G F. 
2 3 



F(x(t),r)) < F(£,y(t)) + + — (a + ln(2 vol(Z)) for all £ G X and 77 G Y, 



Hence, 



which implies by (|15p that 

F(x(t),n) < y(t)) + e for all f G X and 77 G F. 

□ 
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Lemma 4 For any e G (0, 1), there exist a and 
satisfying I7gj) . (Hfy and [T5\) . 

Proof Assume wZ(Z) < |. Let us choose t = Then ((T3J) becomes (after taking logarithms) 

| + J Vln(^)+l n M( Z ))>0. 

So choosing ^ = iVln(^) — ln(uoZ(Z))) would satisfy this inequality. Then 

,' N i 1 1 , 1 
t = O In - + In ■ 



_e 2 e e 2 woZ(Z), 
Since 1/ vol Z < i?^, the claim follows. Let now vol(Z) > ^. Then 

e f( « )iv^ (z)> 1 f ( « )iV 
y Aet J v y 2 Met ; 

Thus, in order to satisfy (|13p . it is enough to find a and t satisfying 

2 e2( 4H } >L 



To satisfy ([To]) , let us simply choose t = |§- + p ln(2 uoZ(Z)) and demand that 

2 e2l 4i? - 2 en ^a+f ln(2^(Z)) j ' 

or equivalently, 



24 \*/ ln(2 uoZrzm^ 

' 1 H ] < ea . 

a 



Thus, it is enough to select a = max |4(ln2 + iVln(f )), 2 v / iVln(2 voZ(Z))} which satisfies 

'24^ a , / ln(2noZ(Z))\ JV m^z)) 
— j <e? and IH - <e « w <e4. 



It follows that 

f 24 24 12 1 fi 

t = max -^(ln2 + iVln(— )), NM2vol(Z)) \ + ln(2 vol(Z)). 

[ £ z £ £ z ) £ 

Since vol(Z) < R N , the claim follows. □ 

Setting T in Algorithm Q] to the value of t given by Lemma 4 yields the following result. 

Corollary 2 Assume X and Y satisfy assumptions (Al) and (A2). Then Algorithm^ com- 
putes a pair of e- optimal strategies in expected 0( n+ ™ fn |r) iterations. 
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5.3 Using approximate distributions 



We know consider the (realistic) situation when we can only sample approximately from the 
convex sets. In this case we assume the existence of approximate sampling routines that, upon 
the call in step [3] of the algorithm, return vectors £ G X, and (independently) r/ £ Y, with 
densities p^(t) and q v (t), such that 



sup 

X'CX 



PX'(t) PX'{t) 



Px(t) px{t) 



< 5 and sup 

Y'CY 



qy>(t) qv'{t) 



q Y (t) q Y (t) 



(16) 



where px'(t) = L eX ' Pt^C (similarly, define px'(t), <lY'(t), qY'(t)), and 5 is a given desired 
accuracy. We next prove an approximate version of Lemma [TJ 

Lemma 5 Suppose that we use approximate sampling routines with 5 = e/4 in step [3] of 
Algorithm^ Then, for t = 0, 1, 2, . . . , we have 

E[$(t + l)]<E[<I>(t)](l + ^ 2 ). 

Proof The argument up to Equation ([7]) remains the same. Taking the expectation with 
respect to r\ (with density proportional to q v (t)), we get 



E 9 [\\p(t + l)||i] < 



Similarly, 



%[|k(t + i)||i]<||? 



i + L _ £ J " eY 

6 2 



£^ £ I^xM t) f veY g v (t)F(£,ri)dr]dt 
+ 6~ + 2 



(17) 



i ?W i 



(18) 



Thus, by independence of £ and rj, we have 

.2\ 2 



E[$(t + l)|x(t),y(t)] < $(t) 



1 + 



6 



e/, . e 2 \ (I^xh( t )L& qr,(t)F($,ri)dnd£ / rle y9fJ(t)/ £e xP€( t )■ F1 (C»»7) d f^^ 



lp(t)lll?(t)lh 



_ e 2 Jggx Pg(*) J^gy %(f)F% y)dr]d^ j v& q v (t) / ggA -pg(t)F(g, 77)^77 

We will make use of the following proposition. 
Proposition 1 If we set 5 = e/4 in $16\) . then 

ftexPti*) f v eY 1r,(t)F(Z,Ti)dTidt J v€Y q v (t) J ?gX Pt(t)F(£, r})d(,d-n 



* 9 



< e. 



(19) 



Proof Since 



neY 



>ZeX' 
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we can bound the L.H.S. of (jl9j) by 



+ 



X, e y q v (t) I^ e xPd t ) F i^v)d^dr ] ^ & q n (t) f^ x p ( (t)F(^,r])d^dr] 



i\\P\t)\\i ll?(tJI|i||PWIIi 
Thus it is enough to show that each term in (|20p is at most |. Since the two terms are 



similar, we only consider the first term. Define X' = {£ £ X : 
1 



px(t) — Px(*) 



} and X" = X\X'. 



\{t)h 



< 



< 



i 



./gex J v eY 
1 f 

Qy(t) J^ex JrieY 



Qv(t)\ F & 

Qr,(t) 



Px(i) Px(*) 
P«(t) P>(*) 



drjdS, 



£ex Px(i) Px(*) 
/ fPg(*) &(*) 
7eeX' VPx(*) Px(t) 

PX'jt) _ PX'jt) . 

Px(t) px(t) 



Px(t) px(t) 

d( 



d V d£ (by(A2)) 



di + 



feX" 



pg(*) _ geft) 
Px(*) Px(*) 

px{t) P x(t) ; - 2 v ^ 



□ 



Proposition [JJ implies that 
E[*(t + l)|a;(t),y(t)] <*(t) 



1 + 



2 \ 2 9 



6 



+ 



1 + 



+ 



< ( 1 + ^ 

36 



The rest of the proof is as in Lemma [TJ 



□ 



Combining the currently known bound on the mixing time for sampling (see |LV041 ILV06[ 
ILV07| and also Section [2D with the bounds on the number of iterations from Corollary gives 
Theorem [TJ 



6 Conclusion 

We showed that randomized fictitious play can be applied for computing e-saddle points of 
convex-concave functions over the product of two convex bounded sets. Even though our 
bounds were stated for general convex sets, one should note that these bounds may be improved 
for classes of convex sets for which faster sampling procedures could be developed. We believe 
that the method used in this paper could be useful for developing algorithms for computing 
approximate equilibria for other classes of games. 
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A Applications in combinatorial optimization 
A.l Mixed popular matchings 

We note that there exist natural examples for which the width parameter p is small. For 
instance, consider the case when F(x,y) = xAy, A G [— l,l] mxn and X and Y are convex 
subsets of the simplices of dimensions m and n, respectively. 

Let us give a more concrete application. Let S,T be two families (say, of combinatorial 
objects), and A £ [—1, l] 5x7 ~ be a given matrix. We assume that these families have exponen- 
tial size (in some input parameter) and hence, the matrix is given by an oracle that specifies 
for each S E S and T E T the value of A(S, T). The objective is to find a saddle point for the 
matrix game defined by A on the set of mixed strategies = {p G : J2s&sPs = 1} an d 
A T = {qeE.l: E Te r^ = l}. 

In general the optimal strategies might have exponential support (i.e., exponential number 
of non-zero entries). However, if the families arise from combinatorial objects in a natural way, 
then the supports of optimal strategies may be polynomially bounded. More precisely, let E 
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and F be two sets of sizes m and n respectively, such that each element S G S (respectively, 
T G T), is characterized by a vector G {0,1}^ (respectively, y T G {0, 1} F ). We assume 
that X = conv{x s : 5 G 5} and Y = conv{y T : T G T} have explicit linear descriptions, and 
furthermore that there exists an m x n matrix A such that A(S,T) = x s Ay T , for all S G S 
and T G T ■ Then it follows from Von Neumann's Saddle point theorem |Dan63j (which is a 
special case of ([I])) that 



min max pAq 

p€A s ggA r 



min max xAy. 

x£X y£Y 



(21) 



Indeed, 



min max pAq 

p€A s ggA r 



mm 

pSA 5 <jGA r 



max psQtA(S, T) = min max psQT% S Ay 



SeS,TeT 



min max psx^A 9ry T = m i n max p$x s Ay 
^ ' SeS Ter 5e<s 

max min T^p^x^^y = maxminxj4y = min max x Ay. 
y£Y peA s ^-^ y&Y x&X xeX y£Y 

S Go 



(see [KMN09 ) . Thus the original matrix game corresponds to a problem of the form ([T]) . 

A special case of this framework was considered in [KMN09] under the name of Mixed 
popular matchings. Let G = (U U V,E) be a bipartite graph, and r : E — > Z be a given rank 
function. A [/-matching M : U — >■ V is an injective mapping such that {(u, M(u)) : u G U} C 
£7. Let S = T = {{(u,M(u)) : u € U} : M is a [/-matching of G} C 2 B . Given 5, T G S, 
define cf>(S,T) = \{u G U : r(u,S(u)) < r(u,T(u))}\/\U\ to be the fraction of the vertices of 
U that "prefer" S to T, and A(S,T) = <f>(S,T) - <f>(T,S). 

It is well-known (see e.g. |GLS93j ) that the convex hull of [/-matchings has the linear 
description X = Y = {x G Rf : J2{u,v}eE x u,v = 1 Vn G U, Y,{u,v}eE x u,v < 1 W G V}. 
Furthermore, if we define ^4 G ]R Sx£; to be the matrix with entries 



^(u,v),{u' ,v') 



if u = v! and r(u, v) < r(u', v'), 







-Tjjy if u = u' and r(u, > r(u', v'), 



otherwise, 



then for any S,T G S, we can write A(S,T) = x s Ay T , where x s , y T G {0,1}^ are the 
characteristic vectors of S and T respectively. Note that in this case p < 1 and vol(X) = 
vol(y) < 1. 

Note that in the above example, the problem can be written as a linear program of polyno- 
mially bounded size [KMN09]. However, this is not the case when the known linear descriptions 
of X and Y are not polynomially bounded, e.g., when in the above example G is a general non- 
bipartite graph. In this case the solution requires using the Ellipsoid method, or the sampling 
techniques of [BV044 [KV06] . or the use of our algorithm. 



A. 2 Linear relaxation for submodular set cover 

Here is another example. Let / : 2^ — > [0, 1] be a monotone submodular set-function. Consider 
the problem of minimizing f(X) subject to the constraint that the characteristic vector ex G 
{0, l} n belongs to a polytope P C R n . For instance, in the submodular set covering problem, 
the polytope P = {x G [0, l] n : Yli-s-3e x i — 1 f° r an e e wnere Si, . . . , S n C E are 
given subsets of a finite set E. Let Pf = {y G R™ : J2iex Vi — /PO for all X C [n]} be the 
polymatroid associated with /. Then it is well-known that f(X) = max y& p f exy- Thus we 



15 



arrive at the following saddle point computation which provides a lower bound on the optimum 
submodular set cover: min xe p max yG p / xy, where p < 1. For other applications of polyhedral 
games, we refer the reader to [Was03| . 
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